Papers with learning model
Automatic Pair Construction for Contrastive Post-training (2024.findings-naacl)
Copied to clipboard
Canwen Xu, Corby Rosset, Ethan Chau, Luciano Corro, Shweti Mahajan, Julian McAuley, Jennifer Neville, Ahmed Awadallah, Nikhil Rao
| Challenge: | Large language models (LLMs) have unprecedented proficiency in a wide array of tasks. |
| Approach: | They propose a way to construct contrastive data using preference pairs from multiple models of varying strengths using SLiC and DPO. |
| Outcome: | The proposed method outperforms existing models like Orca in the comparison of SLiC and DPO with SFT baselines. |
Summarization Evaluation in the Absence of Human Model Summaries Using the Compositionality of Word Embeddings (C18-1)
Copied to clipboard
| Challenge: | Existing summary evaluation methods rely on multiple model summaries to evaluate quality of summary outputs. |
| Approach: | They propose a new summary evaluation approach that does not require human model summaries . they exploit compositional capabilities of word embeddings to develop features . |
| Outcome: | The proposed metric replicates human-generated summarization scores on data from TAC 2008 and 2009 . the features are then used to train a learning model for predicting the summary content quality in the absence of gold models. |
On the Cost-Effectiveness of Stacking of Neural and Non-Neural Methods for Text Classification: Scenarios and Performance Prediction (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural network algorithms excel on Automatic Text Classification tasks, but they are expensive and require high computational costs. |
| Approach: | They propose to exploit the cost-effectiveness of stacking of automatic text classification classifiers to improve their effectiveness. |
| Outcome: | The proposed method can predict the best ensemble in each scenario using only fraction of available training data. |
ALICE: Active Learning with Contrastive Natural Language Explanations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Annotating a large dataset with annotations is costly and infeasible. |
| Approach: | They propose an expert-in-the-loop training framework that utilizes contrastive natural language explanations to improve data efficiency in learning. |
| Outcome: | The proposed framework outperforms baseline models trained with 40-100% more training data on bird species classification and social relationship classification tasks. |
Fˆ2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmax (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text generation do not fully reflect the rich diversity of human language. |
| Approach: | They propose to use F2-Softmax and MefMax to train a balanced frequency distribution using a frequency class-based method. |
| Outcome: | The proposed methods improve the diversity and quality of generated texts. |